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Introduction 


The  overall  objective  of  the  research  is  to  develop  new 
statistical  procedures  for  estimating  item  response  curves  used  in 
mental  testing.  A  typical  test,  which  is  studied  here,  consists  of 
K  test  items  administered  to  n  examinees.  The  data  consists  of  a 
matrix  of  binary  scores  indicating  which  items  are  scored  correctly 
and  which  incorrectly  by  each  of  the  examinees . 

The  analysis  of  such  data  is  based  on  an  assumed  model  which 
specifies  the  probability  of  correct  response  to  each  item  as  a 
function  of  the  ability  of  the  examinee.  Such  probability  models, 
can  be  indexed  by  item  parameters .  The  main  result  of  this  research 
is  the  development  of  new  methods  for  estimating  these  item  para¬ 
meters  for  the  purpose  of  measuring  abilities  based  on  such  estimates 

When  abilities  are  defined  in  terms  of  real  variables  and 
treated  as  fixed  effects,  both  theoretical  and  practical  analysis 
of  item  responses  are  greatly  hampered  by  the  large  number  of 
parameters,  which  increases  as  the  number  of  examinees,  n,  increases. 
The  standard  method  for  simultaneously  estimating  item  and  ability 
parameters  is  maximum  likelihood,  using  standard  programs  such  as 
LOGIST  described  in  Wingersky,  Barton,  and  Lord  (1982).  Some  of  the 
problems  encountered  by  this  approach  are  the  occassional  non¬ 
existence  of  a  true  maximum  likelihood  solution  and  the  lack  of  a 
reasonable  measure  of  the  reliability  of  the  estimated  parameters. 
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The  approach  taken  by  the  current  research  assumes  that 
ability  parameters  are  sampled  from  some  population  distribution, 
which  itself  may  be  indexed  by  parameters.  The  true  likelihood 
function  then  becomes  the  integral  of  the  conventional  likelihood 
function  with  respect  to  the  ability  distribution.  The  resulting 
likelihood  function  is  then  a  function  of  the  item  parameters  and 
the  "hyperparameters"  of  the  unknown  ability  distribution.  Though 
the  resulting  expression  for  the  likelihood  function  is  not  simple, 
it  contains  substantially  fewer  parameters  then  the  conventional  maximum 
likelihood  equations .  A  general  approach  to  finding  the  maximum 
likelihood  estimate  using  the  new  likelihood  function  is  the  EM 
algorithm,  discussed  in  its  general  form  by  Dempster,  Laird,  and 
Rubin  (1977) .  However  the  implementation  of  the  EM  algorithm  requires 
heavy  computation  involving  numerical  integration.  Earlier  versions 
of  the  EM  algorithm  are  given  for  the  one-parameter  logit  model 
by  Sanathanan  and  Blumenthal  (1978)  and  for  the  two  parameter 
probit  model  by  Bock  and  Aitkin  (1981).  Part  of  the  current  work 
is  an  extension  and  refinement  of  these  earlier  applications. 

The  main  work  also  includes  an  extension  to  fuller  Bayesian 
methods  by  introducing  prior  distributions  on  the  item  parameters. 

The  EM  algorithm  can  be  modified  for  computing  posterior  modes. 

The  reciprocal  of  the  negative  second  derivative  of  the  log  posterior 
evaluated  at  the  mode  is  then  used  to  approximate  the  posterior 
covariance  matrix  of  item  parameters .  The  Bayesian  approach  gives 
us  a  means  of  posterior  analysis  and  opens  up  new  tools  for  practical 
problems  such  as  item  selection  and  adaptive  testing. 


•  . 


A  brief  account  of  the  specific  accomplishments  of  this 
research  are  summarized  below.  Fuller  write-ups  of  the  technical 
details  are  given  in  the  technical  reports  and  other  papers  listed 
at  the  end  of  this  report. 

A.  Maximum  likelihood  (m.l.)  estimation  of  item  parameters. 

1.  General  setup. 

When  the  ability  parameters  are  assumed  to  be  a  random 
sample  from  a  distrbution  with  parameter  v,  the  formal  likeli¬ 
hood  function  £(£, y)  becomes  a  function  of  the  item  parameters 
5  and  ability  distribution  parameter  y.  Under  the  assumption 
of  local  independence,  the  EM  algorithm  has  been  demonstrated 

A  A 

as  being  a  powerful  tool  for  deriving  the  m.l.  estimate,  (4.y). 

The  computation  reduces  to  working  with  a  series  of  simpler 

problems  involving  one  item  at  a  time.  For  the  one  parameter 
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logistic  with  a  N(0,o  )  ability  distribution,  the  results  are 
very  similar  to  those  obtained  by  Anderson  (1970)  using  the 
conditional  maximum  likelihood  approach.  Simulation  results 
have  shown  that  estimates  usually  exist  when  they  do  not  under 
the  conventional  m.l.  approach  which  treats  the  ability  parameter 
as  fixed.  Simulations  have  also  suggested  that  there  can  be  a 
savings  in  sample  size  of  10  -  157„  when  calibrating  items  under 
the  one  parameter  logistic  model,  relative  to  the  conventional 
methods.  Similar  results  have  been  found  for  ability  parameters 
estimated  as  posterior  means,  given  (£,y)  =  (£,y).  Details  have 

A(  "w  A<  IV 

been  published  in  Rigdon  and  Tsutakawa  (1983) . 


Convergence  of  the  EM  algorithm  in  item  response  analysis. 

Certain  questions  concerning  the  convergence  of  the 
EM  algorithm  have  been  raised  in  the  recent  literature. 
Convergence  is  guaranteed  for  the  1-parameter  logistic  due 
to  convexity  properties  of  the  likelihood  function.  For 
the  two-parameter  logistic  it  can  be  demonstrated  that 
the  EM  solution  is  the  solution  to  the  likelihood  equation, 
so  that  if  there  is  a  unique  solution  it  will  coincide  with 
the  EM  solution.  Details  are  presented  in  Research  Report  82-1 

Estimation  of  2-parameter  logistic  curves. 

Computational  details  required  in  the  EM  algorithm 
for  the  2-parameter  logistic  model  have  been  derived  and 
illustrated.  The  nonuniqueness  of  the  parameterization 
in  the  2-parameter  model  can  be  eliminated  by  placing 
restrictions  on  the  ability  prior.  In  particular,  for 
normal  priors  on  ability,  uniqueness  is  obtained  by  using 
the  N(0,1)  distribution.  The  asymptotic  covariance  matrix 
of  the  item  parameters  can  be  computed  using  the  empirical 
information  matrix.  Numerical  results  based  on  simulations 
have  indicated  the  reasonableness  of  this  approach  for 
assessing  the  posterior  uncertainty  of  the  m.l.  estimates. 
Details  are  presented  in  Research  Report  83-1. 
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B.  Empirical  Bayes  estimation  of  item  parameters. 

In  certain  situations  the  item  parameters,  in  addition  to 
the  ability  parameters,  may  be  treated  as  a  random  sample  from 
some  prior  distribution  indexed  by  an  unknown  hyperparameter. 

This  situation  arises  when  the  item  parameters  are  exchangeable, 
and  the  prior  information  on  each  item  is  the  same  from  item  to 
item.  Certain  ad  hoc  procedures  have  been  developed  for  the 
1-parameter  logistic  model.  The  basic  procedure  consists  of 
alternately  re-estimating  the  item  and  ability  parameters,  which 
are  assumed  to  be  sampled  from  separate  normal  distributions  with 
unknown  parameters,  until  convergence  is  attained.  Results  were 
very  similar  to  those  derived  under  A.l.  Due  to  heavy  computational 
requirements,  extensions  to  multiparameter  models  were  not  success¬ 
ful.  This  part  of  the  work  is  summarized  in  Rigdon  and  Tsutakawa 
(1984)  . 

C.  Bayesian  estimation  of  item  response  curves. 

The  third  and  final  estimation  procedure  developed  in  this 
project  is  a  fully  Bayesian  method  based  on  a  new  family  of  prior 
distributions  for  the  item  parameter.  This  family  of  priors  differs 
from  the  one  proposed  by  Swaminathan  and  Gifford  (1981),  which 
assumes  that  item  parameters  have  a  common  prior  whose  hyperparameter 
has  a  known  distribution.  The  current  approach  assumes  a  prior 
distribution  on  the  probability  of  correct  responses  at  specified 
ability  levels,  for  each  item  separately.  This  prior  then  induces 
a  prior  on  the  item  parameters.  In  applications  this  approach  seems 
simpler  than  working  through  the  hyperparameters. 
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The  estimation  of  item  parameters  is  made  by  using  the 
posterior  mode  which  can  be  computed  via  the  EM  algorithm.  The 
measure  of  uncertainty  is  then  taken  to  be  the  posterior  co- 
variance,  which  can  be  approximated  by  the  reciprocal  of  the 
negative  second  derivative  matrix  of  the  log  posterior.  Nmerical 
illustrations  for  the  2-parameter  logistic  model  have  shown  that 
the  posterior  modes  are  very  similar  to  the  m.l.  estimator  described 
under  A. 2  and  these  obtained  via  LOGIST.  This  illustration  was 
based  on  a  39  item  math  test  using  a  sample  of  400  subjects.  Details 
are  presented  in  Research  Report  84-1. 


Some  unfinished  work. 

Comparison  of  item  response  curves. 

Two  curves  can  be  compared  in  terms  of  their  logists 
at  different  ability  levels.  In  the  case  of  the  2-parameter 
logistic  model,  the  logits  are  straight  lines.  The  posterior 
probability  that  two  such  lines  are  within  a  given  distance 
over  some  fixed  interval  of  ability  is  presented  as  a  measure 
of  the  closeness  of  two  lines.  This  approach  differs  from 
the  more  conventional  approach  where  comparisons  are  made  in 
terms  of  the  item  parameters.  Numerical  work  based  on  simulated 

data  and  actual  test  data  have  been  completed.  Results  remain 
to  be  written  up . 


■V>.1 
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2.  Goodness- of- fit  study. 

A  cross  validation  study  was  started  to  examine  the 
predictability  of  item  responses  in  one  data  set  given 
observations  on  a  separate  but  related  data  set.  The 
test  statistics  being  examined  turned  out  to  have  power 
against  certain  alternative,  but  not  against  other 
alternatives  which  might  be  equally  important.  This  phase 
of  the  research  was  discontinued  pending  a  better  test 
statistic. 

E .  Summary 

This  research  has  demonstrated  that  estimation  of  item 
curves  with  ability  parameters  treated  as  a  random  sample  is 
a  promising  important  approach  to  item  response  analysis. 

Although  such  modelling  has  been  considered  in  the  past, 
practical  solutions  have  only  recently  become  a  reality  with 
the  advent  of  modern  computer  technology  and  the  EM  algorithm. 

This  research  has  focused  on  the  theoretical  formulation 
and  solution  of  maximum  likelihood  and  Bayesian  estimations  of 
item  parameters.  Algorithms  have  been  developed  and  numerically 
illustrated  for  the  one  and  two  parameter  logistic  models.  The 
results  are  generally  comparable  to  the  conventional  methods  which 
treat  ability  parameters  as  fixed.  The  current  methods  have  the 
advantage  however  of  generally  producing  estimates  when  they  do 
not  exist  under  older  methods.  The  Bayesian  approach  yields  an 
approximation  to  the  posterior  covariance  matrix,  which  can  be  use 
to  make  probabilistic  statements  about  the  uncertainly  of  the 
estimated  parameters. 


Before  widespread  applications  of  these  results  can  be 
realized,  it  is  important  that  user  oriented  computer  packages 
be  prepared.  Such  packages  should  not  only  handle  the  case  of 
n  subjects  and  K  items,  but  must  be  able  to  deal  with  missing 
data  and  other  designs  where  different  subjects  may  be  given 
different  items,  as  in  the  case  of  adaptive  testing.  For  such 
packages,  it  would  be  desirable  to  include  the  3-parameter 
logistic  model,  since  guessing  is  an  avoidable  problem  with  tests 
using  the  multiple  choice  format  which  is  quite  commonly  used  today 

References 

Anderson,  E.B.  (1970).  Asymptotic  properties  of  conditional 

maximum  likelihood  estimators.  Journal  of  the  Royal  Stat¬ 
istical  Society,  B.  32,  283-301. 

Bock,  R.D.  and  Aitkin,  M.  (1981) .  Marginal  maximum  likelihood 
estimation  of  item  parameters:  An  application  of  the  EM 
algorithm.  Psychometrika,  46,  443-459. 

Dempster,  A.P.,  Laird,  N.M.,  and  Rubin,  D.B.  (1977),  Maximum 

likelihood  from  incomplete  data  via  the  EM  algorithm  (with 
discussion) .  Journal  of  the  Royal  Statistical  Society,  B, 

39,  1-38. 

Sanathanan,  L.  and  Blumenthal,  S.  (1978).  The  logistic  model 

and  estimation  of  latent  structure.  Journal  of  the  American 
Statistical  Association,  73,  794-799. 

Swaminathan,  H.  and  Gifford,  J.A.  (1981).  Bayesian  estimation  in 
the  three-parameter  logistic  model.  Research  Report  LR-119, 
School  of  Education,  University  of  Massachusetts. 


10 


Wingersky,  M.S.,  Barton,  M.A.  and  Lord,  F.M.  (1982).  LOGIST  User's 
Guide,  Educational  Testing  Service,  Princeton,  NJ . 


Technical  Reports  and  Publications 
Prepared  under  Contract  No. 

N00014-81-K0265 ,  NR150-4&4 

Tsutakawa,  R.K.  (1982).  Estimation  of  item  parameters  and  the 
GEM  algorithm,  Research  Report  82-1,  to  appear  in  the 
Proceedings  of  the  1982  IRT/CAT  Conference,  D.J.  Weiss,  Ed., 
Department  of  Psychology,  University  of  Minnesota. 

Rigdon,  S.E.  and  Tsutakawa,  R.K.  (1983).  Estimation  in  latent 
trait  models.  Psychometrika,  48,  567-574. 

Tsutakawa,  R.K.  (1983).  Estimation  of  two-parameter  logistic  item 
response  curves,  Research  Report  83-1,  to  appear  in  the 
Journal  of  Educational  Statistics. 

Rigdon,  S.E.  and  Tsutakawa,  R.K.  (1984).  Parametric  empirical 

Bayes  and  Bayes  approaches  to  parameter  estimation  in  latent 
trait  models,  unpublished  manuscript. 

Tsutakawa,  R.K.  and  Lin,  H.Y.  (1984).  Bayesian  analysis  of  item 

response  curves,  Research  Report  84-1,  unpublished  manuscript. 


Ml  SSOUri  /  T  SUt akaua 


1  Dr.  Nick  3onb 
Office  of  Naval  Research 
Liaison  Office,  Far  East 
SFO  San  Francisco,  CA  96503 

1  Lt.  Alexander  Sorv 
Applied  Psycholog/ 
Measureaent  Division 
NAMRL 

NAS  Pensacola.  FL  32S09 

1  Dr.  Robert  Breaux 
NAVTRSESU3PCEN 
Sode  N-095R 
Orlando,  FL  32313 

1  Dr.  Robert  Carroll 
NAVOP  115 

Washington  ,  DC  20370 

1  Dr.  Stanley  Col  Iyer 
Office  cf  Naval  Technology 
300  N.  Quincy  Street 
Arlington.  VA  22217 

1  CDR  Mike  Curran 
Office  of  Naval  Research 
300  N.  Quincy  St. 

Code  270 

Arlington,  VA  22217 

1  Dr.  John  Ellis 
Navy  Personnel  RID  Center 
San  Diego,  CA  92252 

1  DR.  PA*  FEDERICO 
Code  F13 
NPRDC 

San  Diego.  CA  92152 

1  Mr.  Paul  Foley 
Navv  Personnel  PhD  Center 
San  Diego.  CA  92152 

1  Mr.  Dick  Host a* 

NAVDP-135 
Arlington  Annex 
Pool  2334 

Washington  ,  DC  20350 


li-Aug-34 


1  Dr.  Norear.  J.  -e'r 
Chief  of  Naval  Education  and  Training 
Code  o: A2 
Naval  Air  Station 
Pensacola,  FL  32503 

1  Dr.  Leonard  Kroeker 
Navy  Personnel  RID  Center 
San  Diego,  CA  92152 

1  Daryl!  Lang 
Navy  Personnel  RhD  Center 
Santiago,  CA  92152 

1  Dr.  Nilliaa  L.  Malo  -Q2) 

Chief  of  Naval  Education  and  Training 
Naval  Air  Station 
Pensacola.  FL  32502 

!  Dr.  Janes  McBride 
Navy  Personnel  RltD  Center 
San  Diego.  CA  92152 

1  Dr  Nilliaa  Montague 
MPRDC  Code  13 
San  Diego,  CA  92152 

t  Liorary,  Code  P201L 
Navy  Personnel  RhD  Center 
San  Diego,  CA  92152 

i  Technical  Director 
Navy  Personnel  RtD  Center 
San  Diego,  CA  92152 

i  Personnel  i  Training  Research  Group 
Code  442PT 

Office  cf  Naval  Research 
Arlington,  VA  22217 

1  Dr.  Carl  less 
CNET-PDCD 
Building  90 

Sreat  Lakes  NTC.  IL  aOOBB 

1  Mr,  Dre*  Sands 
NC'RDC  Code  a 2 
San  Diego,  3A  92152 

1  Mary  Schrat: 

Navv  ?ers:nnei  PhD  Center 
San 'Diego,  CA  92152 


.  *  -  •  ’  .  ’  ,  *  .  '  *  •  ■  «  .  •  ,  '  ,  ■  .  * .  •  ,  . •  . 


Nissoc'i/TsutakaiM 


!6-Aug-34 


New  fiavy 

;  Dr.  Robert  3.  Saith  1  Dr.  Wallace  Wulfeck,  III 

Office  c f  Chief  of  Naval  Ooerations  Navy  Personnel  RiD  Center 

0P-987H  San  Diego,  CA  92152 

Washington,  DC  20350 

1  Dr.  Alfred  F.  Snode 
Senior  Scientist 
Code  78 

Naval  Training  Eguipaent  Center 
Orlando,  FL  32813 

1  Dr.  Richard  Snow 
Liaison  Scientist 
Office  of  Naval  Research 
eranch  Office.  London 
Baa  39 

-90  New  York,  NY  09510 


1  Dr.  Richard  Sorensen 
Navy  Personnel  RAD  Center 
San  Diego.  CA  92152 

1  Dr.  Frederick  Steinheiser 

UNO  -  opus 

Navy  Annex 
Arlington.  VA  20370 

1  Mr.  8rad  Synioson 
Navy  Personnel  RiD  Center 
San  Diego.  CA  92152 

1  Dr.  Frank  Vi ci no 
Naw  Personnel  RAD  Center 
San  Diego.  CA  92152 

1  D^.  Ronald  Weitzisan 
Naval  Postgraduate  School 
Dsoartaent  of  Administrative 
Sciences 

Monterey,  CA  93940 

1  Dr.  Douglas  Wetzel 
.ode  12 

Naw  Personnel  RiD  Center 
San  Diego.  CA  92152 

1  DR.  MARTIN  9.  »!3ttcF 
NAVY  PERSONNEL  Pi  J  CENTER 
SAN  DIE30.  CA  92152 

1  Mr  .'ehn  H.  Wolfe 
Navy  Personnel  PAD  Center 
San  Di ego,  CA  92152 


Nisseuri/Tiutakawa 
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Marine  Carps 

1  Jen-/  'Lehnus 
CAT  praject  Office 
49  Narine  Caros 
Washington  ,  DC  20330 

1  Headquarters.  Si.  3.  Narine  Corps 
Code  IP! -20 
Washington,  DC  20380 

1  3oec:al  Assistant  Tor  Narine 
Cjros  Natters 
Code  lOON 

JT’ice  of  Naval  Research 
300  N.  Quincy  St. 

Arlington,  VA  22217 

1  Na-cr  Frank  fohannan.  liSNC 
Headquarters.  Narine  Corps 
iCcde  IP! -20 
Washington,  DC  20380 
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Aray 

2  Dr.  Kent  Eaton 
Any  Research  Institute 
500!  Eisenhower  31  vd. 

Alexandria  ,  VA  22333 

1  Dr,  Classen  Martin 
Aray  Research  Institute 
5001  Eisenhower  Blvd. 

Alexandria,  VA  22333 

1  Dr,  Williaa  E.  -torlDrocx 
FNC-ADCD  Box  25 
AFO,  N>  09710 

1  Dr.  Harold  F.  O’Neil,  Jr. 

Director,  Training  Research  Lab 
Aray  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Coaaander.  U.a.  Aray  Research  Institute 
for  the  5et.avi oral  i  Social  Sciences 
ATTN:  PER1-8R  (Dr.  Judith  Orasanu) 

50C1  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Nr.  Robert  Ross 

U.S.  Aray  Research  Institute  for  the 
Social  and  Behavioral  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22313 

1  Dr.  Robert  Sasaor 
U.  S.  Amy  Research  Institute  for  :ne 
Behavioral  and  Social  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Joyce  Shields 
A r«v  Research  Institute  for  the 
Behavioral  and  Social  Sciences 
500!  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Hilda  Wing 
Aray  Research  Institute 
5001  Eisenhower  five. 

Alexandria.  VA  22333 
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Air  Farce 

1  "r.  carl  A.  Alliiisi 
HS.  AFHRL  IAFSC) 

Brocks  AFB.  TX  73155 

!  Nr.  Raynond  E.  Christal 

afhrl/nce 

Brooks  AFB,  TX  73235 

1  Dr.  Alfred  R.  Fregly 
AFQSR/NL 

Selling  AFB,  DC  20532 

!  Dr.  Patrick  Kyllonen 
AFHRL/MOE 

Brooks  AFB,  TX  73235 

!  Dr.  Randolph  Park 
AFHRL'NGAN 

Brooks  AFB,  TX  78235 

1  Dr.  Roger  Pennell 

Air  Force  Huaan  Resources  Laboratory 
Lowry  AFB.  CD  30230 

2  Dr.  Nalcols  Bee 
AFHRL.’NP 

Brooks  AFB.  TX  78235 


Deoartient  of  Defense 

12  Defense  Technical  Information  Center 
Caaeron  Station,  Blog  5 
Alexandria,  VA  22314 
Attn:  TC 

1  Dr.  Clarence  HcCcraick 
HE,  HEPCOM 

MEPCT-P 

2500  3reen  Bay  Road 
Noorth  Chicago,  1L  40064 

2  Military  Assistant  for  Training  and 
Per sonr,e2  Technology 

Office  of  the  Under  Secretary  of  Def 
for  Research  4  Engineering 
Rooe  3D  129,  The  Per.iagen 
Washington.  DC  20301 

1  Dr.  if.  Steve  Sellnan 
Office  of  the  Assistant  Secretary 
of  Defense  !NRA  t  L? 

2B24?  The  Pentagon 
Washington,  DC  20301 

1  Dr.  Robert  A.  Wisher 
OUSDRE  (EL31 

The  Pentagon,  Room  3D1C? 

Washington,  DC  2030! 
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Civilian  Agencies 

1  Or.  0atricia  A.  Butler 
N'E-BIW  Bldg.  Step  4  7 
1200  i?th  St.,  m 
Washington.  DC  20208 

1  Dr.  'fern  K.  lirry 
Personnel  R4D  Center 
Office  of  Personnel  lanageaent 
1900  £  Street  !W 
Washington.  DC  20415 

!  «r.  Thosas  A.  liar* 

J.  S.  Coast  Suard  Institute 
p.  0.  SuDstation  18 
Gklahoua  Cit*,  OK  73 IS? 

!  Dr.  j’oseah  L.  Young.  Director 
fleaorv  «  Cognitive  Processes 
National  Science  Foundation 
Wasnmgton,  DC  20550 


Private  Sector 

1  Dr.  Erling  B.  Andersen 
Department  of  Statistics 
Studiestraede  S 
1*55  Copenhagen 
DEN«AP.< 

1  Dr.  Isaac  Bejar 
Educational  Testing  Service 
Princeton,  NJ  08450 

1  Dr.  ffenucha  Birenbau* 

School  of  Education 
Tel  Aviv  University 
Tei  Aviv.  Ranat  Aviv  d9?'3 
Israel 

1  Dr.  Werner  Birke 
Personalstasmast  der  Bundeswehr 
D-5000  Koeln  90 
NEST  SERMftNY 

1  Dr.  R.  Darrel  1  Bock 
Department  of  Education 
University  of  Chicago 
Chicago,  II  S0S37 

1  dr.  Arnold  Bohrer 
Section  of  Psychological  Research 
Caserne  Petits  Chateau 
CRS 

1000  Brussels 
Belgium 

1  Dr.  Robert  Brennan 
Aeerican  College  Testing  Programs 
P.  0.  Box  ia8 
loua  Cit/,  IA  52243 


1  Dr.  Blenn  Bryan 
iZ08  Pee  Road 
Bethesda.  HD  20317 

1  Dr.  Ernest  R.  Zadotte 
30’  Stokeiv 

University  of  Tennessee 
Knoxville.  TN  F?li 


1  Dr,  John  B.  Carroll 
409  Elliott  Rd. 

Chapel  Hill.  NC  2751* 
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Private  Sector 

1  Dr.  Noraan  Cliff 
Dept,  at  Psychology 
Univ.  of  So.  California 
University  Park 
Los  Angeles,  CA  90007 

1  Dr.  Hans  Croebag 
Education  Research  Center 
University  of  Leyden 
Boerhaavelaan  2 
2334  EH  levden 
The  NETHERLANDS 

1  Lee  Cronbach 
16  Laburnua  Road 
Atherton,  CA  9 '■SOS 

1  CTB/NcGraw-Hil!  Library 
2500  6arden  Road 
Ncnterey,  CA  93940 

1  Hr.  Tiaothy  Davey 
University  of  Ulinhois 
Department  of  Educational  Psychology 
Urbana,  IL  61601 

1  Sr.  Dattoradad  Divgi 
Syracuse  University 
Departaent  of  Psychology 
Syracuse,  NE  33210 

!  Dr.  Eaaanuel  Donchin 
Departaent  of  Psychology 
University  of  Illinois 
Chasoaign,  IL  61320 

1  Dr.  Hei-Ki  Dong 
Ball  Foundation 
Rooa  314,  Building  B 
800  Roosevelt  Road 
Glen  Ellyn,  IL  60137 

1  Dr.  Frit:  Drasgou 
Departaent  of  Psychology 
University  of  Illinois 
603  E.  Daniel  St. 

Chaspaign,  IL  61820 

1  Dr.  Stephen  Dunbar 
Lindquist  Center  for  ileasureaent 
University  of  Iona 
Iona  City,  .A  32242 


Private  Sector 

1  Dr.  John  H.  Eddins 
University  of  Illinois 
252  Engineering  Research  Laboratory 
103  South  Natheus  Street 
Urbana,  IL  61801 

1  Dr.  Susan  Eabertson 
PSYCHOLOGY  DEPARTMENT 
UNIVERSITY  OF  KANSAS 
Laurence.  KS  66045 

1  ERIC  Facility-Acquisitions 
4833  Rugby  Avenue 
Bethesaa,  HD  20014 

1  Dr.  Benjaain  A.  cairbank,  Jr . 
Perforaance  Metrics,  Inc. 

5825  Callaghan 
Suite  225 

San  Antonie,  TX  7322B 

1  Dr,  Lecnard  Fslat 
Lindquist  Center  for  Heasuraent 
University  of  loua 
Iona  City,  IA  52242 

1  Univ.  Proy  Dr.  Serhard  Fischer 
Liebiggasse  5/3 
A  1010  Vienna 
AUSTRIA 

1  Professor  Donald  Fitzgerald 
University  of  Net*  England 
Araidale.  Neu  South  bales  2351 
AUSTRALIA 

1  Dr.  Dexter  Fletcher 
University  of  Oregon 
Department  of  Coaouter  Science 
Eugene,  DR  97403 

1  Dr.  Jon n  R.  rrederiksen 
Bolt  Beranek  V  Neman 
50  Noulton  Street 
Cambridge,  HA  02138 

1  Dr.  Janice  Sif'ord 
University  of  Hassachusetts 
School  of  Education 
Aaherst,  HA  01002 
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Private  sector  Private  Sector 


1  Dr.  Robert  olaser 

Learning  Research  Si  Developient  Center 
University  of  Pittsburgn 
393?  O’Hara  Street 
PITTSBURGH,  PA  15260 

•  Dr.  Harvin  D.  clock 
217  Stone  Hall 
Cornell  University 
Ithaca.  NY  14853 

1  Dr.  Bert  Green 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  6  34th  Street 
Jaltuore,  HD  21213 

1  DR.  JAHES  G.  SPEENO 
lRDC 

UNIVERSITY  OF  0 1TTSBUKEH 
3939  O'HARA  STREET 
PITTSBURGH,  PA  15213 

1  Dipl .  Fad.  Hichsel  A.  Habon 
'jniversitat  Dusseldorf 
Er i i ehungswi ssenshaf 1 1 1 cries  Inst.  II 
Uni/ersitatsstr.  1 
3-4000  Dusseldorf  I 
WEST  SERhANY 

1  Dr.  Ron  Hasbleton 
School  of  Education 
University  of  Hassacfiusetts 
Aiiherst,  HA  01002 

1  Prof.  Lutz  F.  Horr.ke 
•Jniversitat  Dusseldorf 
Erciehunqswissenschaftiiches  Inst.  II 
‘Jniversitatsstr.  1 
Dusseldorf  1 
WEST  GERHANV 

1  Dr.  °aul  Horst 
«:t  3  Street.  #134 
Chula  Vista,  CA  90010 

1  Dr.  Lloyd  Huaohreys 
Dejartaent  of  Psychology 
University  of  Illinois 


1  Dr.  Steven  Hunka 
Depariaent  of  Education 
University  of  Alberta 
Edaonton.  Alberta 
CANADA 

1  Dr.  Jack  Hunter 
2122  Ccolidge  St. 

Lansing,  HI  43906 

1  Dr.  Huynh  Huynh 
College  of  Education 
University  of  South  Carolina 
CoJuabia,  SC  2920B 

I  Dr.  Douglas  H.  Jones 
Advanced  Statistical  Technologies 
Corporation 
10  Trafalgar  Court 
Lawrenceville,  NJ  09148 

1  Professor  John  A.  Keats 
Departaent  of  Psychology 
The  University  of  Newcastle 
N.S.W.  2308 
AUSTRALIA 

1  Dr.  Williaa  Koch 
University  of  Texas-Austin 
Neasiire.ment  and  Evaluation  Center 
Austin.  TX  78703 

1  Dr.  Thoeas  Leonard 
University  of  Wisconsin 
Department  of  Statistics 
1210  West  Dayton  Street 
Hadison,  HI  53’ 0! 

1  Dr.  Alan  Lesgold 
Learning  RiD  Center 
University  of  Pittsb'Tgr 
3"?  O'Hara  Street 
Pittsburgh,  PA  15260 

1  Dr.  Hichael  Levine 
Deoartaent  of  Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Chaipaig.n.  IL  61801 
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Pri/ate  Sector 

1  Dr.  Charles  Lewis 
Faculteit  Sociale  Wetenschappen 
Riiksuniversiteit  Groningen 
Dude  Boterinaescraat  23 
9712SC  Groningen 

'  Netherlands 

1  Dr.  Robert  Linn 
College  of  Education 
University  of  Illinois 
Urbana,  1L  61301 

1  Dr.  Robert  Lockean 
Center  for  Naval  Analysis 
200  North  Beauregard  St. 
Alexandria,  VA  2231! 

1  Dr.  Frederic  N.  Lord 
Educational  Testing  Service 
Princeton,  NJ  03541 

1  Dr.  Oases  Luasden 
Deoarteent  of  Psychology 
University  of  Western  Australia 
Medlands  W.A.  6009 
AUSTFALIA 


Private  Sector 

1  Dr.  Melvin  R.  Novick 
356  Lindquist  Center  for  leasuraent 
University  of  Iowa 
Iona  City,  IA  52242 

1  Dr.  Dales  Olson 
WICAT,  Inc. 

1873  South  State  Street 
Ore#,  UT  84057 

1  Navne  M.  Patience 
Aierican  Council  on  Education 
3ED  Testing  Service,  Suite  20 
One  Dupont  Cirle,  NW 
Washington,  DC  2003s 

1  Or.  Oases  Paulson 
Dept,  of  Psychology 
Portland  State  Univevrsity 
P.3.  Box  751 
Portland,  OF;  97207 

1  Dr.  Mark  D.  Reckase 
ACT 

P.  0.  Box  163 
Iona  City,  IA  32241 


1  Dr.  Gary  Marco 
Stop  31-E 

Educational  Testing  Service 
Princeton,  NO  08451 

1  hr.  Robert  McKinley 
Aae-ican  College  Testing  progra«s 
P.0.  Box  163 
lea  City,  IA  32243 

I  Dr.  Barbara  Means 
Huaan  Resources  Research  Organization 
300  North  Washington 
Alexandria,  VA  22314 

1  Or.  Robert  Mislevy 
Til  Illinois  Street 
Geneva,  IL  60134 

I  Dr.  W.  Alan  Nicewander 
University  of  Oklahoaa 
Department  of  Psychology 
Xahsia  City,  OK  73069 

h' 


1  Dr.  Laurence  Fudner 
403  Els  Avenue 
Takes a  Park.  MD  20012 

1  Dr.  J.  Rvan 
Department  of  Education 
University  of  Soutn  Carolina 
Columbia,  SC  29208 

1  PROF.  F'JMIKO  5AMEDIMA 
DEPT.  OF  PSYCHOLOGY 
UNIVERSITY  OF  TENNESSEE 
KNOXVILLE,  TN  37916 

1  Frank  L.  Schmidt 
Deoartment  of  Psychology 
Bldg.  5G 

George  Washington  University 
Washington,  DC  20052 

1  Louell  Schcer 
Psychological  i  Quantitative 
Foundations 
College  of  Education 
University  of  Iowa  •  ! 

lo.a  City,  IA  52242  "H 


Nissouri/Tsuiakawa 


:6-fiug-d4 


Private  Sector 

1  Dr.  Kazuo  Shigeeasu 
7-9-24  Kugenuaa-Kaigar 
Fujusawa  251 
JAPAN 

1  Dr.  Nil  1  lie  Siis 
Center  for  Naval  Anal '.si  5 
200  North  Beauregard  Street 
Alexandria,  W  22311 

1  Dr.  H.  Wallace  Siraiko 
Prog rae  Director 

Manpow  Research  and  Ad-isor*  Services 
Saithsonun  Institution 
3il  North  Pitt  Street 
Ale.andm a.  -Iff  22314 

1  Nartha  Stocking 
Educational  Testing  Service 
Princeton,  NJ  00541 

1  Dr.  Peter  Stolaff 
Center  tor  Naval  Analysis 
200  North  Beauregard  Street 
Alexandria,  V A  22311 

:  Dr.  Wiliiae  Stout 
University  of  Illinois 
Ceoartsent  of  Nathesatics 
'Jr oar. a.  1L  61301 

!  Er.  Hariharan  Swaeinathan 
Laboratory  at  '’sychoeetric  and 
Evaluation  Pesearch 
School  of  Education 
'universit,  o*  Massachusetts 
Aanerst.  NA  019)3 

Dr.  Kinuai  Tatsuoka 
Coaouter  Based  Education  Research  Lab 
23*  Engineering  Kesea-ch  Laboratory 
L'raana,  ll  61301 


rage 

Private  Sector 

1  Nr.  Sary  Thosasson 
University  of  Illinois 
Departsent  of  Educational  Psycholog 
Chaipaign.  IL  61820 

1  Dr.  Robert  Tsutakawa 
Department  ot  Statistics 
University  of  Missouri 
Coined! a.  NO  65201 

1  Sr.  Ledyard  Tuci.er 
u',i«ersitv  o;  Illinois 
Department  :f  5svcnolcav 
603  E.  Camel  Street 
Chaipaign,  IL  61320 

1  Dr.  V.  s.  R.  upouluri 
union  Carbide  Carccratior 
Nuclear  Division 
p.  5.  Be.;  t 
Oak  Ridge.  TN  37S30 

1  Dr.  David  Vale 
Assessaent  Systees  Corporator. 

2223  University  Avenue 
Suite  310 

St.  Paul.  NN  5*114 

1  Dr.  'Howard  ««:ner 
Division  of  Psychological  Studies 
Educational  Testing  Service 
Princeton,  NJ  08540 

!  Dr.  Mirg-Nei  dang 
i-indouis:  Center  for  Messu'esent 
j!’ly9rs:tr  of  Iona 
Iona  City  ■  1A  57242 

1  Dr.  Brian  Haters 
HuoPFG 

300  North  Washington 
Alexandria,  VA  227 14 


1  Dr.  Maurice  Tatsuosa 
220  Education  Bldg 
1310  S.  Sixth  St. 
Chaipaign.  IL  61320 

l  Dr.  David  Thissen 
Deoartaent  of  Psychology 
University  of  Kansas 
Laurence,  KS  66044 


1  Dr.  David  1.  Weiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  NN  55455 

l  Dr.  Rand  R.  Nil  cox 
University  of  Southern  California 
Department  of  Psychology 
Los  Angeles,  CA  90007 
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Private  Sector 

i  Genan  Military  Representative 
ATTN:  Wolfgang  Wi 1 degrube 
Strei tkraef teaat 
D-5300  Bonn  2 

4C00  Brandywne  Street,  NW 
Washington  ,  DC  20016 

1  Dr.  Bruce  Willi ais 
Department  of  Educational  Psychology 
University  of  Illinois 
Urbana,  1L  61901 

1  Ms.  Mari  1  yn  Hingerskv 
Educational  Testing  Service 
Princeton,  NJ  0334 1 

1  Dr.  Secrge  Hong 
Siostatistics  Laboratory 
Memorial  Sloan-Xettering  Cancer  Center 
1275  York  Avenue 
New  York,  NY  10021 

1  Dr.  Nendv  Yen 
CTB/NcGra*  Hill 
Del  Monte  Research  Park 
Monterey,  CA  93940 


